The lindner example, Overview

https://thomaselove.github.io/500-examples/#the-lindner-example

The lindner data come from an observational study of 996 patients receiving an initial Percutaneous Coronary Intervention (PCI) at Ohio Heart Health, Christ Hospital, Cincinnati in 1997 and followed for at least 6 months by the staff of the Lindner Center. The patients thought to be more severely diseased were assigned to treatment with abciximab (an expensive, high-molecular-weight IIb/IIIa cascade blocker); in fact, only 298 (29.9 percent) of patients received usual-care-alone with their initial PCI. The lindner data is part of the twang package.

The analysis provided here was developed by Wyatt P. Bensken and Harry Persaud. It involves 12 separate tasks using methods for both quantitative and binary outcomes that are analogous to those in the toy example developed by Professor Love. The main difference is in style, and in the fact that 1:1 matching without replacement performs very poorly.

The lindner example, Introduction

The lindner example does 1:1 greedy matching on the linear propensity score, in this case however there are more treated subjects than controls, so the default greedy approach taken is to not match all treated subjects, since you run out of controls before you get to them all.

• 1:1 nearest neighbor matches are demonstrated, first without replacement (in Task 4) and then with replacement (in Task 6).
• The “with replacement” matching has some appealing features, not least of which being that now we can realistically create ATT estimates (since we match all treated subjects to a control, repeating some controls.)
  • The match quality (in terms of a Love plot) is much better with replacement in this case.
  • This does come with a cost, though. Most subjects are matched just a few times, but some are included more than 50 times!

Lindner Data

• 996 rows, 12 columns

• Sample: 996 patients receiving an initial Percutaneous Coronary Intervention (PCI) at Ohio Heart Health, Christ Hospital, Cincinnati in 1997 and followed for at least 6 months by Lindner Center staff.
• Quantitative Outcome contained in cardbill
• Binary Outcome contained in sixMonthSurvive
• Treatment Status contained in abcix
• Seven Covariates: stent, height, female, diabetic, acutemi, ejecfrac and ves1proc
• Somehow, no missing data.

Lindner Outcomes and Treatment

Stratified Outcomes

• I’ve rearranged some factors so they’re a little easier to think about, as compared to the lindner example online.
• Linear Model: Treated subjects estimated to spend $1512.46 (95% CI: -3.83, 3028.76) more (unadjusted analysis).
• Logistic Model: Odds of “alive after 6m” in treated subjects are 3.31 (95% CI: 1.51, 7.48) times higher than controls (unadjusted).

After Matching: Outcome Results

• After 1:1 matching without replacement (698 matches: lousy covariate balance)
  • so we didn’t look at the outcomes
• After 1:1 matching with replacement (698 matches: good covariate balance)
  • In the matched sample, mean cardbill for treated was $16,127 and was $16,300 for controls.
  • Treated subjects were estimated to spend $-173.03 (95% CI:-1482.29, 1136.22) as compared to non-treated subjects.
  • The odds of “alive after 6m” were 5.6 times higher in treated subjects (95% CI: 2.86, 10.98).

Key Outcome Results

After subclassification into 5 quintiles:

• Treated subjects estimated to spend $732.67 more (95% CI:-876.5, 2341.85) than controls.
• The odds of “alive after 6m” were 4.33 (95% CI: 1.68, 11.19) times higher in treated subjects.

After ATT (Average Treatment Effect on the Treated) Weighting

• Treated subjects estimated to spend $239.28 less (95% CI:-3019.92, 2541.37) than controls.
• The odds of “alive after 6m” were 6.5 (95% CI: 2.27, 18.65) times higher in treated subjects.

Key Outcome Results

After direct adjustment for the linear propensity score:

• Treated subjects estimated to spend $1167.90 more (95% CI:-412.22, 2748.02) than controls.
• The odds of “alive after 6m” were 4.64 (95% CI: 1.99, 11.27) times higher in treated subjects.

After ATT Weighting and direct adjustment (double robust):

• Treated subjects estimated to spend $126.72 less (95% CI:-2514.24, 2260.80) than controls.
• The odds of “alive after 6m” were 6.9 (95% CI: 2.28, 20.84) times higher in treated subjects.

1:1 Greedy Matching

Greedy (or nearest neighbor) matching selects a treated subject and then selects as a matched control subject the untreated subject whose propensity score is closest to that of the treated subject. If multiple untreated subjects are equally close to the treated subject, one of these untreated subjects is selected at random, typically. Options include:

1. Select treated subjects from highest to lowest propensity score.
2. Select treated subjects from lowest to highest propensity score.
3. Select sequentially treated subjects in the order of the best possible match.
   • First treated subject is the one who is closest to an untreated subject.
   • Second treated subject is the one closest to the remaining untreated, etc.
4. Select treated subjects in a random order. Set a fixed random number seed so that the matched sample is reproducible in subsequent analyses.

Results in all treated subjects being matched to a single control.

Greedy Matching with Replacement

Matching without replacement means that once an untreated subject has been matched to a treated subject that untreated subject is no longer eligible for further matches to other treated subjects. As a result, each subject can be in at most one matched pair.

Now, in matching with replacement, we allow members of the “control” pool to be reused in the matching process.

• The process is somewhat simpler in the nearest neighbor case - just match each treated subject to the closest untreated subject.
• Because untreated subjects are recycled and thus can be included in multiple matched sets, the order in which the treated subjects are selected has no effect on the formation of matched pairs.

Caliper Matching

Match subjects only if they fall within a pre-specified maximum distance (the caliper distance.)

• When using caliper matching, we usually match subjects on the logit of the propensity score using a caliper width as a proportion of the standard deviation of the logit of the propensity score.
• Caliper matching can be combined with other distance metrics (where, for example, a few specific covariates are targeted for more precise matching.)
• Matching with a caliper can be accomplished with or without replacement, and in 1:1 or 1:k settings.

Matching without Replacement

We first consider methods based on matching without replacement. Using this approach, we matched each untreated subject to at most one treated subject. Once an untreated subject has been matched to a treated subject, that untreated subject is no longer eligible for consideration as a match for other treated subjects.

• The primary distinction between different matching algorithms that use matching without replacement is between optimal matching and methods based on greedy matching.
  • Optimal matching forms matched pairs so as to minimize the average within-pair difference in propensity scores.
  • In contrast, greedy nearest neighbor matching selects a treated subject and then selects as a matched control subject, the untreated subject whose propensity score is closest to that of the treated subject (if multiple untreated subjects are equally close to the treated subject, one of these untreated subjects is selected at random).

Caliper Matching

A modification to greedy nearest neighbor matching is greedy nearest neighbor matching within specified caliper widths.

• In this modification to greedy nearest neighbor matching, we can match treated and untreated subjects only if the absolute difference in their propensity scores is within a prespecified maximal distance (the caliper distance).
  • When using caliper matching, we matched subjects on the logit of the propensity score using a caliper width that was defined as a proportion of the standard deviation of the logit of the propensity score.
  • We match on the logit of the propensity score because the reduction in bias due to the use of different caliper widths is better understood.

Matching with replacement

Matching with replacement permits the same untreated subject to be matched to multiple treated subjects.

• We considered nearest neighbor matching with replacement and nearest neighbor matching within specified caliper widths with replacement.
• Each of these approaches simply matches each treated subject to the nearest untreated subject (subject to possible caliper restrictions).
• Because untreated subjects are recycled or allowed to be included in multiple matched sets, the order in which the treated subjects are selected has no effect on the formation of matched pairs.

Austin’s Case Study

Sample of 9107 patients discharged from 103 acute care hospitals in Ontario, Canada, with a diagnosis of AMI (or heart attack) between April 1, 1999 and March 31, 2001. We collected data on these subjects as part of the Enhanced Feedback for Effective Cardiac Treatment (EFFECT) study, an initiative intended to improve the quality of care for patients with cardiovascular disease.

• For the current case study, the exposure was whether the patient received a prescription for a statin lipid-lowering agent at hospital discharge. 3049 (33.5%) patients received such a prescription.
• The outcome was a binary variable denoting whether the patient died within 8 years of hospital discharge. 3593 (39.5%) patients died in that window.

Case Study: Propensity Matching

We estimated a propensity score for statin treatment using logistic regression to regress an indicator variable denoting statin treatment on 30 baseline covariates.

• We used restricted cubic smoothing splines to model the relationship between each of the 11 continuous covariates and the log-odds of statin prescribing.
• We used each of the matching algorithms under study to form matched samples consisting of pairs of treated and untreated subjects.

Details on Bias Reduction

• In the original sample, 10 of the 30 covariates had standardized differences that exceeded 0.10, with the largest standardized difference being for history of hyperlipidemia (0.88).
• Optimal matching and the nearest neighbor matching without replacement algorithms resulted in substantially improved balance. In the matched samples, 29 of the 30 covariates had standardized differences that were less than 0.10. The standardized difference for hyperlipidemia remained large (0.43) in these matched samples.
• The four algorithms based on caliper matching without replacement resulted in substantial reductions in imbalance: all standardized differences were less than 0.101. Of these algorithms, caliper matching (closest distance) resulted in the best balance (all standardized differences were less than 0.04).

Austin’s conclusions re: 12 Algorithms

• Larger numbers of matched pairs (from complete optimal or complete greedy matches) yields more precise estimates than smaller numbers of matched pairs (say, when a caliper is used and only some treated subjects are matched.)
• Caliper matching often yields better “balance” and less biased estimates as compared to other algorithms.
• So we have a bias - variance tradeoff in our estimation strategies, but in terms of MSE, caliper matching usually performs pretty well.
• In terms of ordering of treated subjects for greedy matching or caliper matching, random selection is competitive with other options.
• Optimal matching is pretty comparable to nearest neighbor matching with random selection order, and in fact, it’s not clearly any better than that approach.

Funding and Acknowledgements

Dr. Ahmed was supported by the National Institutes of Health through grants (R01-HL085561 and R01-HL097047) from the National Heart, Lung, and Blood Institute and a generous gift from Ms. Jean B. Morris of Birmingham, Alabama.

“The Beta-Blocker Evaluation of Survival Trial (BEST) study was conducted and supported by the NHLBI in collaboration with the BEST Investigators. This manuscript was prepared using a limited access dataset obtained by the NHLBI and does not necessarily reflect the opinions or views of the BEST Study or the NHLBI.”

The authors of this manuscript have certified that they comply with the Principles of Ethical Publishing in the International Journal of Cardiology.